Weakly Supervised Semantic Labelling and Instance Segmentation

Semantic labelling and instance segmentation are two tasks that require particularly costly annotations. Starting from weak supervision in the form of bounding box detection annotations, we propose to recursively train a convnet such that outputs are improved after each iteration. We explore which aspects affect the recursive training, and which is the most suitable box-guided segmentation to use as initialisation. Our results improve significantly over previously reported ones, even when using rectangles as rough initialisation. Overall, our weak supervision approach reaches ∼ 95% of the quality of the fully supervised model, both for semantic labelling and instance segmentation. Example Output after After After Ground input rectangles 1 training round 5 rounds 10 rounds truth Figure 1: Example results of using only rectangle segments and recursive training (using convnet predictions as supervision for the next round), see Section 3.2.